CS261: A Second Course in Algorithms Lecture #12: Applications of Multiplicative Weights to Games and Linear Programs∗

1 Extensions of the Multiplicative Weights Guarantee Last lecture we introduced the multiplicative weights algorithm for online decision-making. You don't need to remember the algorithm details for this lecture, but you should remember that it's a simple and natural algorithm (just one simple update per action per time step). You should also remember its regret guarantee, which we proved last lecture and will use today several times as a black box. 1 Theorem 1.1 The expected regret of the multiplicative weights algorithm is always at most 2 √ T ln n, where n is the number of actions and T is the time horizon. Recall the definition of regret, where A denotes the action set: max a∈A T t=1 r t (a) best fixed action − T t=1 r t (a t) our algorithm. The expectation in Theorem 1.1 is over the random choice of action in each time step; the reward vectors r 1 ,. .. , r T are arbitrary. The regret guarantee in Theorem 1.1 applies not only with with respect to the best fixed action in hindsight, but more generally to the best fixed probability distribution in * c 2016, Tim Roughgarden. 1 This lecture is a detour from our current study of online algorithms. While the multiplicative weights algorithm works online, the applications we discuss today are not online problems.